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be thang- that  deomet seer reasonable: a) that a single respemse t 


2 De te Ite is pt orertoenmtble pPasis tor evaluation, Gitnosis, or 
Presevrptienm,y amt bh) that the fact that S is wrongs at the Sire ti 


sieMittes coy he rs wrens, Tt maw be a little slower going, but ome 


amoooasainee through Crowder<tvpe prograns simply by throwing darts at 
Fess se atheermitivess it 1s dike plaving a stot machine that: oop 
mis Voor Py vith O sivenifsing boop and 1 signitviny advance. tue 


spellings’ preg rat co be sketched will be interactive on a serhentetoe 
see ent, rither than ap item-to-item, basis. We will characterize 
seePe nts as oor 2-20 iter" length, with abrupt termination of tae 
eeeent @onosatbrlits vhenever ats items require Ptev-te-tter resr 3 
throu. the kevhoaurd and on-line ovalWation of perfercince. 


PoUMa pernigs student-sv tem rnteractive studbes usin one er oa 
eolinetteonoot the tollowing presentation rodes: Video Only (WO), vie 
Mis CA), Lidia + Video (CAV). Spelling instruction entirls that rin 


rosa stooaddboe ederents have duratrans on the order of Po second--the ti ¢ 


phot are) feoroaonounee werds of ones or twoesublabde fen th. Tt uiltoow 
Catoosa thet the GP)lustrative spelling anit: ter the 6 part ce al. 

rt? settings * opeeavilabie ws ord... foe dGilustrataive sregra cri 
Petra t AW orters during inetructional segments and SO rte ss dura 


t «f soy, rts that with compre atrens te be reve dts), Sore Cr rarst 
vootrectie nab ptems wild require Fe cboard responses.  Poepenses ts 


ry-trecthponmal ates will be evatuated fror oa standpoint of ipprenriaten 

Popdedn. ane ote tetercime whether pereentiibarechanp ab ohidle isc d 
trout ft rtoftath trem ope preaqdde tha re.  Peaponses to test ites ryt 
et oalteate Pou gine to ape DDanm. evatrerna OCohieh wae dit '. rn Ceneqabinme oat 


eta A ek > ee ne . Potala thes report 8 4 bye . 
Ss ee Oe OS oc | ot yr 4 mab tet Dts 
fe hel ye Jb ee oe Getty Foie “hee PT a Ped trotl ‘ , 
Peet oboe go tate bbemed prove rate ta Sopa og eA ag Ge CEE Gea, ae ga ’ 
per 8 pee Mi hays gh tas ag yee Pe Aja eae ty ae GR Bget gt, A 
“4 Nove apy Perrbater toootet ft woah orb bpetiu. tb mooie ts 
boat oe ter rest te Spettaer pote. er a ‘ “oot that om fey 
= hot oof amstria tan tora shh ov. Doe mtart crm entatren rf 
ra : Pte ou rete bday eo bona bh og mete ee not dee rena rec, 
ny Taro to omresentaitron at the serhes cs wish be ee eta? ate © bs FT 
Rag Poet ert ha Yt legate thy t t eeooret abo aintert. OY «a afte ; 
ter pata te t FSS 0 TOO Pa OE be 
soele cent. set tame ea te the tee toe terthad souprees foe 
Py og abesrerey reg seeeryipsyy ir ar ee es or hoa! reeds dere daopms 
re we tge ss that sete doa thee  acurees Ohypedy eta tore 
ror de ate bbb oar yo the ager rectertmapay aetpebde abtem pa thet tos 
ee nsw ts aoorat to Uoatoondy sata Lome ropes 
Poem tosh oot oudte e be tent. eee ebb he oamite vrort. 
roeoatt bore a gbade tae eb ooat ttm tp oe be eee me arate pt. bo 
"OR yhappaa Poa® ertary foe Phat katate qayeer ta 
art 
cop 41 P40 OY Ceti Pet pb te at daistite feed oe Medites, iF 
Pep ete Vt tee tf sho etter oh fead-et foompita, bth ¢taaisa et pet, 
COT Pese bottom vonttued te the ty sf sert)enta. SEE enon, 
eat GP obtstre tromad segment cto ite  theredm caw recite mil 
Womsesesbeth colertoame omert. bp ome Wishes, one fan 
eat Sta ete oo darim. oreserptatden of instructional rter, 
chete Sh Gem apd rate ter ates present atpem., Or cme ie ck bine 
Cadet. ob ee tee At respons. that orerpdect . elecent . of aba dl : 
srilled test vecponses.  Ebdustretite ino tractronil oeycent 
ROA port. ie ot ot thoes pore gnire du itetben Cer copyrna 
referenmcdpe foe the Voudesent during ede bo present etaer, 
Solpowime uresemttger ot sped or seat enty the sveter will ; 
, ertesoet orestomses aarmat ¢oerdte rion tor aerptabaitite. 
testes the oveeter vabboselert a osteentertvimible terpothe ot 
Ghar yg Teese apes pelo opnetra trol settware. [tra 
tagtooga athe homely pt fount paspehede y will ressudt peoope rere 
nitahte De ot I Saar ant f Ste faethe nea ati ce, 
Sy Fae Presse riptive coy cteit rete retyee do ter pp traetiegar * 
ny oat | Pouce tee Gea dba cease foe eat om etre feat team? 
tosotw, 
eprom oboe tepb atts ve tabhe ent qad pres. ah berm at 7208 
od hae A 4 may made ob torres Meee met bye cer gdete. So rt oa 
mee padem oan bee ree bre to beter Poa? peta. attra ova d aby 
See bodem unter gthbon, Pareto ompes amt rtd) tf 6 gb oan tees ' 
faeet Gp ayy aro ay weer tees pe Ta a dat Pe dy oat be Se tat ata 
later once ventotder bo hey mare nt teh? aap ae ee oe do | 
ife ae t ae porte ta ef a Py 4 eh thy pe oo bho | 
| 
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Po sth eo Poste edb borte. be, : ‘ ; Hoa ‘ a 
(Tres Imi pre.nrs Prldoorut  obabo or, fet be ew te Hy Y 3 
mig se POPE as Se tea : ae 
tye ape Se ey rot tory Yay rte . my 4 , \ 
Mie me remee t rate iH t ee oe iH ' ae ' ' . 
ze eat dr Gfoan ea gare Pak at Hee 
. te STs hits Ses Pe OP to ae note ; 
ata, ready ate trode give ‘ et ee Reach EY a a : 
ye he AGP bar Ta Pose ate to eee date woe m * tds he 
iv ets Chat Getta. soe yt abe DY ae, Tea eg 
emanse a te ‘ t So, EE I is 4 in Te Hy) j Rec pe af 1 a | 
i thee Sato atte sf fee 8 oe eb ob, Py ent tratbomed tea. ot : 
hast. Gf sata om oy T abt Pore pone. ha reer tos ’ 
@ * eothev we but dno. pebete , tee dy teats eae odie Lae tf a 
Seevent Por ore entast peta at or slaw r pea. & pe Oy il. 
Pe re dectedy Sut treet da totete Co bemet abate at dys sport. root 
. ormaccentabte fret i af ort ieee Pee foes Neds 
(ee ouresstned ale ta rem oo. ‘hp sort y pote cetia bea besa 
heoodtaarptte Peo bet ue cum oe tut the eee | ‘ ferdao. 
Secaapple thane tobe bast Petre r gn to ten Poaty ue, datter 
a a ne er a 2 a er ed 
a a a 
: Treat puns fa ve gt Ta ie oleae tts e4os “tot Ss 
tober apd redtee to rate a, Uh pg ph atb om ped bovre ag fy : 
foodicates the —Inetrmetaomiab aerent Potota tetwn theca oe po! : 
Pistang throat oer pas den- 6 stem alow t eyye cent pit ee ®t | 
Tuet dettar Swat Pep). Eage thy 3 i ta ota tee 4 ‘ ‘ | 
46 Present Pare Pop Wha Ge Sey ow Yad erie dar ‘4 Lt at Peta da. as ‘ ’ 
che chatet tes atten te atl letter Pte bor Seep uetpe t it tte | 


werd as Tale, this cast de aeeot plished be bbeok ia. bbb 


rea git Poet the out oe 


revpre ses. Svan fe the mest pte: eettps tedtewtira the 
pressing of Plo tha. sevrent vtieht be teroamete d either ' 
weeeptabidite ctiterpen ter the oeerabl recpen ot , 


toterpre peo ea bhatt fe. aye cry pe’ Pea 
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ve oP PeSe mt Bi Piast se awk Seated ® 1 a -ie ' Vet ow that 
perforcanee ago, oe Pat athe ee ta waye Peta 
Perism ania 
Present Te to cannact > rool. t sped dae ae eae oe hy oes ot? 
hWeesarts of te ct ite... dere hee fiat fpacit he ': aot tir? 
test fer ospelbing . to pre,rar werd Owords whose snedlliraes re 
eopbed dtrdme anstreet ion), sbpeh combo des qe. aueb betes fade 4 


On the basis et retrbevinge spedldime 1 


ror recor, and Db) the. 


whaivb test fer veneraligather of oe darpe rules dy usim 


fest rtoers mevel words (wel 


Ee Consonant with sye llains : is 


teoeptable perporwance retere sm te 


Wetels shrobab boawes Gat dere ot nresereata 


duriny trotrietdeomr) tat 
ter dntredueea.,  tn- 


pros ram oamed tee moe Tooraire 


baat mad Vom r ate tte fomse. pore py Partie sc. Thy ot rhe Beg ty 
pte oe Abuation-commind-switebing sequence of anterssibs bavet bin 
tos fbom is relative. Sometiees 20 seconds could te allowed; ooretis 


WO elas weold pe unaicseptel te. 


Pree value of 


eat he @opime doen the rathe et oar ote: down ttn. 


Poe tree spent by the. ster presenting atoms 


Phat obey ap thee os presentallon-resporye tite. 


wy 4 } . 
Bass Pa Pe 


aE OP OED a ag mado ee ute thew vere ond. (ts coalt 


detas between present atien 
P tite mere sbrnitte 


and aerepting, respon oe 3 


Down time pe stulent- 


y dined be the eye ter tee nabinite the situatien, beau 


Yo geeessing afters). ada 


Edy trative dostructremi) @oragin Cese., opelling, phontes) invites tes 


sentation cet ospeceh temeth. Reo ponses shomtbd ooo mere thea 


eee ope mab ds 0 samee unaintemoted sped dim mo teatare rather thigh ites 


reoertittbonoctenmeadties per oundt tate, be dear t! 


et 


bby seme od speedy YS tbe od ebotired an 
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boaNaten yg long, other 


lige od sth de mot byes. Me do net ask tor switching in oon 


1077, we rake the opparentt: 


a) 


| , 7 tt 
Tetsu bbe Cero? thet Che own pertiben fb a .hert downe-up trial’ mt 
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PON Dates rte our obich 


SU] giatineg om opresentatten ed og Vo frame ot oa CRE to secur in | 


Tess teddbowrna deteetaum ot tee ceptrediang 


Wee bote at a doyneup ort be that ds favor, ble 
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ett Ret Che Sequence ob opermaf bon 


dierted code cumder od! 


et the cwestorepossibd: eave sae conditions, phe video svste: ah 0g 
toads Nom th ooh opemtup ee te Grae panmeon gecess freer PRIA" rare betes 
fs UF date 

sha tbe fae arpa oe eed aes Tare at op pos cthbe te he conte = 
Siate Dormteractarve wore tindess Qhdbe te tru ramdoi: ae cess bre tone 
tee ceive prevaded fer PDOMS cides but obarageterizina beth bdew tad 
tboay then we preboabbs wid) eed Guactent fhe ceeteranee, trrou.dy 


wlattrem te oatoet oeribowetorr audi. ster ape that parable ds 


Tate dthout do vin the ovnehrenieaitien feature 


fretbtad Coste . ae oo tatent wath radi re ivr os, 

PoP 2 oreetabry o aly tora to the beved 
*y 
ad 


theat tate ee 
robe rent oan tutpines 
However, the adda trata 


thre totg a it ee ee ) 


tothe leved of the agudie element. tven though the ilbustrative provram 
Pyomet trivialiv shorty ee will areue tater that the riaximum number « 
such sexnents vequired will be far short of 1800, that audio storage cap 
handle the tllustrative pregram's audio component, and thet we can 
tolerate the relatively slow segment-aecessing times so long as the 
ratraseyment interitem down times are of short duratron, Given that 
seument of interest is in S's audio butfer, we ask the svstem to step 

tx a following rtem | secend tollewing completion of the response to the 
item that comes betore ft and to initiate video frame accessing by the 
end of that second. Assuming, then, a rather large intersegment 
lecessime tame, it still weuld be true, if intrasegment and interitem 
Jown tapes were averaged, that trial down time would average ta less than 
> Seconds for adie and fess than 3 seconds overall. Hence, the 
rtlastrative dewn interval tor inetructional items fs taken as 3 seconds. 
‘ceptabilityvoef an interval of this magnitude depends on magnitude of 
the apo omnterval, 

Pet audio eoue up oon earphones and video come up on the CRT at the 
Seaamminse ef a 4th cecond. Audio will eo off at the end of the 4th secon, 
Ceneeivably, there will be vounger children whe could perform a $s or 4 
letter ipitatien response through the kevboard during Sth and 6th second. 
It ose, then these fastest children retlect a worst case (a d-sivmd above 
the pean case). A median-paced child more likely will use 4 seconds 
(Sth threugh Sth) to complete such responses, Cunder system encouragement 
tt that). Henee, we nivht think of the presentation-response interval 
a bern 3 seconds under worst-case instructtonal conditions and 5 seconds 
tyothe average. Thus, werst-case dtem consumption rate fer Wo fterms 
tdi he PO oper cimute fper terminal). The average ter consunmptbon rate 
ter such atems will be 7-S rer minute (per tertinal). 


lesatipe Peatures (Oo rede presentation, Hence, Voretrieval tire 


ros edt during testing, reducing down time ta f seconde. Lr the 
Wsyabentoas correct that the spelling response should require nere tboe 
than the dortation response (Toar not sure thet at is), then the SeeOnG 
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sere’ trom down tare might be used turing testing te prolong the up 
intersal. (Phere as little ,oednt in relasing A iter retrieval tire 
Curing testing unless dt can be shown that different Subsystems migit 
Sretitatde be emploved, deoeondenag on whether Wor AQ mode applres.) 


notreaction probably will oon occasion be panetuated with Little 


Such Speeehes sboutd improve the down-up ratio; whether the 


Ino an dinotructsvonalde prodim. tive sense rs unknown. 
lore picture sketehed above vields a worst ¢. Psboratie for SVound 
Pr? roatie for AQ; tt vields in averagescase Lt Peratie tor Woand ts 
rata, ter 40, (should one wish to evaluate probable f-terminal Cratty. 
under normal distribution, ao jesipma below the mean case could be viewed 
as ousipy three tires as much presentatron-response fiee vis wor t cise.) 


ete we fave veots date the future--fhat is, rt oedargowteal aopli- 
_ Poveda Sater, Tras been prette se db oe .toblished in conscque rns 
eutcing EPCMS te oc tensive uSes-then it might be agreed that raitros 
less tiverabie te cup tine might suttice. However, LMS 1s the 
Pmstrupent one will ue ta evaluate such matters. We cannot test toward 


the upper linits fer speeded onstruction if hardware is allowed to fis it 


the empirical attack seo ods toe prectuce such tests, 


Poregeim: rerarks establish desirability of a worst-case item 
consucption rate of 10 per minute. reterenced te one terminal.  Maynitude, 
yt the storage requirement underlying suppert of such a rate remains te 
be determined, 


Randow \cess Requirerents. When one has a large collection o! 


rreres aay one of which one may wish te display at any time, then trie 


tendon accessing to such items 1s required. Tf items of the collection 
de net class under a mnemonically-useful linear or hierarchical taxoner is 
schome, then the sort ef sterage we see in Video Master Dise becomes 
revutred, "ven if mnemonicaullv-useful classification is possible, it has 
no useful implications for Video Master Disc storage when this is used 

in cennection with [DCMS termi .als, because multiple usership of Petts 
precludes a movable head ‘staving in one place over item presentations 
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referencing to a piven S, Thus, 1t would be no advantave to group vido 
trames physteally on i dise just because they are likely te be used in 
the sice seement of instruction or even in a fised order in that senment. 


Joon we transter over to Audio Master Reproduction (the audi stored, 
it atarm becomes ditticult to see how a classification scheme could are 
eqyubprent response time per user under the condition of true random 
tooeasang from oa large number of alternatives, However, the audre tin 
eo video Subsystem ditfer im one amportant respect. \udie Butter oop 
Feeedve oad semanmee ot aidie elements that as physically grouped on 
Vadie “Master Reproduction Cabthough Toanm not sure that suvhow tisece- 
echemeoe seroent then ean be mide fe run as the audio portion of a series 
PooS pte sentatbons whese pace ds controlled by the syste, at looaist 
not until citterent oftware 2s previded than appirentiv curreptty | 
sentesplated be the centraeter). Video Suffer can only receive one 
Thave at oa tate. date sertse of classification fer the audie os eter caiht 


ere ard, 


Secure ofarst that chi dren will be oat oer near the care peeadet at 
treocottset of the anestrae ronad prewram but at ditferent points [ates 


Poth ye bare ditterent user. entering Cor wishing te cater) the s.r 

fone tronspert oot Sudie “aster Reproduction at the same tire, this raivht 

pecescitate basing 2-3 copies ef the “first quarter” of the provera 

moapdabhe rn Aneéiro “aster Reproduction at the outset--thator., 4-6 

Prop eneerts mvt centarno oie cepy of the tarst quarter of the presre 

Caner PE? trans, rt. are availiabley. En consequence, during thi oh 
beataaetaen 2 4 terminals raght be tied te the 4-f transports cont iiarte 


@ aD woo.) theo anetractrenal segments for the phase. OCueufboay teres 


© thee Sto oor ad paeent oer nt ahd) be a prelter at the outset 
: ' 


mda * ther VP eR bee aa ee Povddawesyee our antaeb pre, thee 
St bes sopres heature ot storaure teculd drop out. Mhopeatter, the adgtte rept 
trapoports “eit be ose? ge Tinedr oreanivatien te ref Tect tae prost= 
qgeetul lanetrooreainte ito of res odnine seement .  Vbether carl! or Lote. 
veoemts crebt be stored vecordins te a svete wlpoh ofmimives nr lai tel 
ademin? te adidpe tane transports. 

Vesti’ oseeottd thaitowe ft coments of anetructien eam be asser bled 
OY tebine ahead of rnstrouctien adaoconeeddence of futbs -explieated: decay rs 
Sevedormrent specifications. That bs, ttems Of a oocement can be ordered 
rthent reaneet te perterranve data and, ff alternitive orders tre 
rreoatred, then differentiv-ordered but otherwise identical copres of ty 
« “ett can be usserhled, avata witheuat repard to performance, Mas ped 
sot ream that a seement must ran its course. The segment coutd fe 
Perramated at amy point in consequence of S reaching a upper or lors 
tilerren hound detaaed thrown application of Sequential analvsie. tu 
rinertant questions then heeome a) How many andia sequences and rearcers ; 
Donres ot these sequences would anhere in the lareest pertron of the 
provram that necds to be stered on a worst-case dav? ob) WIE} re ab tue 
weates te issemble sneh sequences (when properd© istructed) on-Tine’ 


hardier conourptien rate assumptions warrant the wiew that Day 19 
Vit te the a pareeaee tur Yar Tiuetrative instr tions At the one 
roar To, the pastest child wild complete the provram, the average eaddd 


& bata he 23 throneh, am? te efowest enild perhaps 1/3 threngh. Hence, 
Coysater qe. steravn rs bo $oof toe program Cadthouen citheat vhotesorern am 


Tae ent ow fo audio secuence sterage deterrieney s,s svotem resnor ee ceed 
lol Pee queda tape oyster peoused., TP ove assume sequences wild run on 

toe oorfer of 2 oeenutes an the worst case, taen ve mreht well he willing 

to totetoate tor abaoste20 cevend rewind time at tables aay tape transport 

fa Pew oe to erther on? at 6 tape and te transfor 190 tneles of 

Shor qa ote Mplie Pofferss a cor to ease ter steraeve position. What remains 


tor ote determ pr. as hore rat osm sedttemces wall ooceur an the terminal 


pede BP fe oe TL 8 a og 


ie 8. Gibte w tlhe ioe ye! peter teed) pale om The Tye best cla ii'y 


ae itor td te ty oamute ss boas, fae dedine centrat gnstruetronal pats 

Pfeat ot ome ts pas Teta meatoe thar thr child sal] Toep one toon 
Be ae gy Ne AO Ges parte preacry pedals 300 Cemainute semmentso., Of thes, 

Poy thor at ye between maisndine and boops aprertions 740 segments t 
omtoe aun #O te doops. Tet ou interpret this as 120 instruct boas 

sie, Ge TTY wy OO Sesto ent, ]> instruetiensreterenced diarnost ie 

eooent oy bo onatrr Jiomereterev:cd proeseriptive sepments, 1D teot- 

vetorenocd pre riptive ses oente, and Po abfernative test sepnents. fet 


saan that othe aeeraee ebadd wil? show a dr) retre between patnd ine 


mf doops, necessitatine appertioning BO sesrents fo loops. turther, let 
fog en Pores toe ddd wid) wre ay Crd ret. between vedas 
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Hehe. horst cme oo ar Boe. 


Tecoundjeated above frome cared da Pipe my wer ddoa ete 8 yp Papaya Pe 
hater too the Poopine sermept sterirm requpreteat. oot tote a oe,tr 
ferns. tor present purpeses, Petous assume thet thes dearest sft 
hooves they presorietive, ane adternmative test sents. are lb mog ta 
Ppoeb of serments that ors trdee the e1ze useda- et sconente. 2 


mused seroents Ghich eisht be oused py other user yore ft) re bee qu 
eooompet ee sure qust what sey ents wild be nerded. Tt ose fe ral set 
Dh ou ero peweaded that seme are never used, the conld bev detest. 


toaneeroeld)y ee perrence mrebht abso shou that ner oe rtete mere sectbe res 


feo Worstecde: ola Car TO), the stowest corbd 2s appre i. tte 
(oS taureueh the progr and the rastest child as com odetine the neces 
esnmine chat the boorine Jevel is constant aero s the preyramc-proh al | 
PWorst cases-then on Dav 19, the system would need to ster: just on 
eee tf (2? ¥ 4BOV/3, er 320 sepments of dipitaliv-r coded aati materi 
efor ae sme 2O drertalleecoded andro elemerts per semeent pins hater 
rooneeded te aderess the audio portion of the segrent. Pet ts aesine 
tet each segment can be «tered on apprev imately V7 day tes eb tape 


Cy Seeones . be 73 rps, vath dieetal codes occurrin? deed? oot tuegr ame. 


@ Ptoe oon the elas sitiedatton assamptiron made eairbrer, © rs gsi! 


foto the students necver vill wo te the same Audio Master Reproduct fon 
tae transport at the same tive. S's own audte bufier contains tt 
tee otransperts. Le oa sume here that while one of these tap trium-snrt 
Peotmouse, the er stem alsars will antietpate the ~most-probable net 
cessentoand canse at teohe delivered te the other tape transport herr 
tre oharine serment or conupleteds At wor)et, then, the ovstem will 
Ptitess be trone. Onls ain thew inetarecS till anteractrve instraet: 
sitter sorething: on the order of of 20-second versatcease rowindetrinster~ 
ye oord-rerdnd penalt'.  sueh a penalty should be encountered noo more 
toan | time an 3 for the slowest child and To time in > fer the taste t 
ob ose ff the sestem vere operating on the basis of charee ratin 
heey on the basis at oprier pertero ance by Sy including on the een 


Meoproae otpated, Pe surmarize, the mean such penabt: weld ive 
pro drated Tw cede, experienced one time in four foes te. 
Paves REA 
Sty Saster Peproduction ceansists of & medudes, cach contabmias 


tape trauspert . baeh transports reelecentered on 4 tect oot tam 
ee Mormad=pla speed 1. 3.79, with badirectrenal drice over either 
tro tracks. Pach transpert will take oo? minutes of andro mater sa 
coach adt sid) hieh-specd search at Paps, bach tract im ceithe 
Crro tion from ceel center measures 2740 inches. Tard tore eae 


PT gs ee ys Tape) each teroack dn oettier dire fren cam stare apore toate 
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: , : fe ies a Po tame trapeport oy tabs ap se 


toragee Edd gece odate the vorst— ase dav stories rewadrerent 


Ih ope ar. poe rhde that phi ins a daurve number oof smadi preerae 
erante on ot fane wreht necessitete minor mediticitron of co ster 


rian’ ar oftyare, te svetem teptealls enplave a "2 pragma’ per 
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\wain, we assume that CCVC and CVCC eveles will have the same evste 
bordeningn characteristics for [rain-l through Train-5 as does the CVC 
‘cle. Training materials will not be the same for letter-sound and 
bipartite patterns; the same items will he used but segmented and blended 
differently in terms of units used. For the present we assume that the 
ditferent strategies will be consonant with training coverage illustroted 
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sove for the | tter-sound pattern strategy. 


Posttests will test for the same skills as do entrv skills tests 

& cept that novel rule words will be used. Because the letter-sound rule 

et will censist of only 46 items after Cvcle 1 training, increasing to 

I? items after Cvele 2 and to 18 items after Cycle 3, possibilities far 
nneratine novel rule words will be meager following Cycle 1 training 
indo preater with completion of training on succeeding cycles. Hence, it 
tS assumed here that there will need be fewer items in the CVC posttest - 
tian in later ones. While this could be remedied by teaching the most 
rules duying Cvele | and the least rules during Cvcle 3, there is no 
hic te approach the experimental design problem that is entirely 
itisfactors., Here it is asSumed that Cycle | posttests each will 
eptiin 4 items, Cvcle 2 posttests 7 items, and Cycle 3 pusttests 10 
ttems. The assumption is that posttests will use the same amount of 
t'oe per item as do entrv tests. 


the tixed seument view typically would not sit well with VF, 

itioularl’ for testing seements but, to a lesser extent, also for 
traiminy seuments. before determining how many alternate random-orde red 
erosions of a segment might be emploved, let us [nventoryv segments.  liere 
itt be JTS test segments and 3 posttest segments for each evele, or 
‘ys itterent test seements. There will be 5S training segments for cach 
sele for each strategy, or 39 different training segments. Since 

ments will be of different lengths, it is necessary to calculate vedi 
toraze based on segment lengths. We will assume that macimum times 

iven above entail that amount of audio storage, which usually will not 
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Sb inutes storage per tape transport is consonant with 9 randomiy-or: 
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room cinates in all. Maximum times for synthetically trained proups 

ri oeroups, each t= 12), by cvele, will be 20, 23, and 26 minvtes, or fu 
routes in all, Maximum times for combined analvticallyv-synthoticaliv 
rained groups (2 groups, each N = 12), by cvcle, will be 22, 25, and ?* 


rnutes, or 75 m1nutes in all. Assuming use of 20-minute periods only 
6)  inutes of which was usabic, then under the conditions of 4-terminal 
toe Of LUCMS, one } per terminal, and good administrative support 
& re, erdips seheduling of Ss, the study as illustrated could be accomplished 
wsim. tla avstem 4 hours per dav (9-12, 1-2) for 5 Successive sthaol 
Pos. Assuming appropriateness of the character generator for all video 
ero rptoa, the study materfals could be constructed without recourse 
teoartists or other printshop personnel. If we made it a requirement that 
‘re ovsterm Cand augmenting equipment) be used for purposes of constructiny 
site rtals only during normal down time (e.g., before 9 a.m., and after 
.» during lunch hour, on weekends), then perhaps a single technicsan 

vai construct, organize, and load in everything needed to support a 

tree oof the sort outlined above in 8-16 hours without hindering svstem 
© eboftatien toc research purposes. Poubling that figure to provide 
Sor che skins and editing and doubling it again te accommodete - tudy- 
tails red production of sottware (if that proves needed), the svstem might 
need to be used on the order of 48 hours during its down time for purposes 
‘ supporting study execution and 20 hours for purposes of executing the 
tudy und performing whatever analyses were required. the personnel bill-- 
wo dasive of study formulatien and write-up--would be 48 technician hours, 
P20 research assistant hours, and perhaps 20 IDCMS supervisor hears, Mv 
~edietion is that this would represent at least a fivefold savings on 
Daf the study actually will cost when conducted using mobile laboratory 
awetlities at the schoels and pre-IDCMS capability for providing graphic 
in} analysis support. We need to do better than that; [| suspect that we 

eld do appreciably better the second time around. 
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ABSTRACT 
A manipulandum-referenced taxonomy for response categories 
appropriate to primary education is presented. The tenability of 


automatic on-line evaluation of the different types or response when 


J processing equipment is of the sort that most probably will be 


v 
available to SWRL during the next year or so is preliminarily 
evaluated. 
‘ 
US DEPARTMENT OF HEALTH 
— EDUCATION & WELFARE 
NOTIONAL INSTITUTE OF 
EDUCATION 
tee fia oe NT NAS HEE be & Peet) ra 
pork tr Bane t vy as ECE wht FOAM 
Thap PR ON CW OR ANIA AT ON OE UNE 
ati Tory Nt eb vib Ch OPEN TONE 
tate De OT NEE AWE Y WE PHF 
ENT GEE Oo AE TION, LING TIT TE OF 
Pree ON TOM RPO Y 
«> ; 
>» | Aes . 
Ww = é 1 eat te tatended ' or internal stoft distribation and use Fermis¢ion Co reprigt of paste from this » tes us 
ERIC ] J pasty atoll roan part, cbowdd be obtained from SWwRE, E1300 la Clenega Boulevard, Inglewe (, cabiteraia, boa 
oi 
L + 


ne 


ON-LINE IDCMS EVALUATION OF DIFFERENT CATEGORIES OF RESPONSE 


Joseph F. Follettie 


Whether on the basis of its own Version 1 capabilities or in 
consequence of augmentation based on core-to-core interfacing and use 
of supplemental terminals, we wish in time to transform the SWRL 
Instructional Development Control and Monitoring System (IDCMS) into 
one whose student-system interactive capability is not bounded by 
the category of response employed. This paper presents a taxonomy 
for response categories appropriate to primary education and pre- 
liminarily assesses tie degree to which we can expect such a system 
soon to perform interactively as a function of response category. 

If we can visualize a relatively straightforward way of arming the 
system both to accept responses of a given category and to evaluate 
them on-line against a criterion for response acceptability, a 
preliminary view can be taken that the system has or can be made to 
have an interactive capability referencing to that response category. 
Otherwise, such capability will be realized only following extensive 
and perhaps costly effort. Unsurprisingly, the general conclusion 
is reached that those response categories whose responses cannot yet 
be on-line machine cvaluated pose a greater challenge to linguists 
and semanticists than to those whose efforts reference to hardware 
design. 

For present purposes, the taxonomy of response categories can be 


viewed as grounded upon three mutually-exclusive and, taken together, 


exhaustive superordinate categories. These are: a) motor responses 


e° 


using characters or codes (keyboard-referenced), b) voice responses 


(microphone-referenced), and c) spatial motor responses. In this 
system, motor and voice categories contrast. Hence, a visual tracking 
response would class as a motor response, since it clearly is not a 
voice response. Physiological responses below the molar motor level 


are not of present interest. 


Motor Responses Using Characters or Codes 


wv 
IDCMS is most clearly applicable to student-system interactive 


use when the response is formed on a keyboard containing alphanumeric 
characters and supplementing commands and codes. Three subcategories 
of such responses are distinguished. 

Surface syntactic constructed response. Any phrase or sentence 
or unordered, semiordered, or ordered set of phrases or sentences which 
a test item invites and given instruction, if effective, constrains 
will class here if made through an alphanumeric or similarly coded 
keyboard. Sequential responses to complex arithmetic problems--e.g., 
response sequences which show steps enroute to problem solution--also 
class here. On-line processing of a surface syntactic constructed 
response presupposes existence of an effective parsing program (and 
perhaps augmenting programs addressing semantic and set-relational 
problems). Oversimply perhaps, the system will have to reduce the 
response to a "base" form and to compare this base form with a cri- 
terion base form response retrieved from an auxiliary store. While 
on-line processing of responses of this type are hardly a "no sweat" 


proposition, SWRL's Language Analysis Package effort currently is appre- 


ciably addressing the problem. It appears not imprudent to assume that we 
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will be able to impute to IDCMS an on-line evaluative capability 
regarding an appreciable portion of such responses before too much 
longer, whether on the basis of Version 1 configuration of the system 
or in consequence of augmentation of the system's processing capability 
using a suitable auxiliary processer. 

Alphanumeric item constructed response. Responses to simpler 
arithmetic problems, word-spelling ‘esponses, and item or concept 
naming responses will class here. Whatever the criterion response 
characteristics regarding accuracy or speed, on-line evaluation of 
such responses appears to pose no insurmountable challenge to soft- 
ware designers. Moreover, it should be possible to cause the system 
to model or shape such response sequences by progressively unblocking 
the next correct keypress in the sequence and, if desired, backlighting 
it. The alphanumeric item constructed response is the type of con- 


structed response that the system can be made to on-line evaluate most 


easily. 

Selected response. Constructed motor response options are 
constrained only by instructional effectiveness and surface alternatives 
inherent in syntax or mathematical structure. Selected response options 
are more narrowly constrained by the test item. The response must be 
selected from a sharply-constrained field--e.g., the alternatives of 
a multiple-choice item. Typically, such responses will be made by 
depressing a single key, whether alphanumerically coded cr color or 
picture coded. Typically also, a portion of the keyboard will he blocked 
off for use in making such = cepanace The keyboard originally proposed 
for IDCMS was such a reduced keyboard--one consonant with making selected 


responses to multiple-choice items. 


r- 
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The selected response need not be restricted to a single keypress. 


Some interactive systems compel S to use a sharply-reduced vocabulary 
and syntax for purposes of interacting with the system. Often in such 
situations, the effect is to restrict S's response to one of a few 
alternatives--e.g., CORRECT, WRONG, DON'T KNOW, Even though S may have 
to learn the perceptual-mechanical skills underlying loading such 
responses into the keyboard, the responses themselves class as selected 
rather than constructed in the present context because a constructed 
response is constrained only by instructional effectiveness and struc- 
ture of an interaction language of nontrivial complexity. That is, 

we call responses constructed only if inettective instruction makes 

a large set of response alternatives entertajnable. If the set of 
entertainable response alternatives must be small even though instruc- 
tion is ineffective, then the response classes as selected. On-line 
evaluation of selected responses made through a keyboard appears well 


within IDCMS capabilities. 


Voice Responses 


The speech and speech element responses classed under this heading 
parallel motor responses using characters and codes excepting that 
input into a "written-form" analyzer must be output from a speech 
analyzer. This heading also subsumes trained voice responses~-whether 
in the sense of diction or of music, 

Surface syntactic speech response. Evaluation of speech responses 
at the syntactic level presumes everything that evaluation of motor 


responses at this level presumes and, in addition, a front end analysis 
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performed by a speech analyzer that transform spoken into written form. | 


It is probable that on-line evaluation of speech responses at the 
syntactic level will not be performed by IDCMS either in Version 1 
configuration or in an augmented configuration that we can obtain 
soon. The alternative is to have E rather than the system perform 
real-time evaluation and to signal the result to the system. To the 
extent that E can do this quickly and reliably, then the E-augmented 
system gives us on-line evaluative capability regarding such responses. 

Alphanumeric item speech response. Comments made immediately 
above apply. Letter sounds form an important subclass of the responses 
that class here. While on-line evaluation of responses of this sub- 
category by IDCMS seems a less challenging requirement than system on- 
line evaluation of speech responses at the syntactic level, here too 
the prognosis for automatic on-line evaluation very soon is not an 
optimistic one. Again, the E-augmented system can be used to achieve 
on-line evaluation during earliest use of IDCMS. 

Selected speech response. This mode of responding simply is 
alternative to motor responding through a keyboard under the selected 
response condition. Having Back a capability might prove desirable 
if it can be obtained cheaply. As an alternative to on-line evalua- 
tion using the E-augmented system, SWRL engineering staff might seek 
to design a sharply-constrained speech analyzer (if a cheap shelf 
item does not exist) which would discriminate between the voiced 
items of such series as 1, 2, 3, 4, 5 and white, black, red, green, 
pink. Such a device really would be a speech pattern discriminator 


rather than a speech analyzer. 


Trained voice respons}. Whether this is a diction or music type 
————— : 

response, the essential feature of the response is that it should match 
a standard. Perhaps eis simply comes down to a comparison of two 
frequency x amplitude plots, one provided by S and one contained in 
system storage, with degarture from standard guaged against allowable 
departure yon standard, It is probable that early capability for 
on-line evaluation of such responses comes down to how finely differ- 
ences between response and standard are to be discriminated. SWRL 
engineering staff has invested preliminarily in the problem. It 
appears tenable that on-line evaluation at least of certain types of 


musical response couid be obtained early in the life of the system. 


Spatial Motor Responses . ~- 
@ We distinguish between drawing and tracking spatially-referenced 


responses and between "freeboarded" and "keyboarded"” drawing responses. 


( Freeboard~referenced drawing responses. The RAND Tablet and 


‘, 


\gevices currently in prototype at the University of Illinois, University 
of Pittsburgh, and elsewhere exemplify requiref hardware. Functionally, 
this is a terminal whose response area is a flat two-dimensional surface 
such that stylus-writing on the surface will bring that portion of the 
surface into contact with a sensing surface. Such devices permit copying 
responses, free form drawing responses, and intermediate responses. 

The mechanical analogue, for those unfamiliar with these devices, is the 


"blackboard" toy wherein a child applies a wood stylus to waxed paper 


over a black tarlike material. Following composition on such a device, 


it is returned to the preresponse state. 


§ 


id If a pattern is placed on the writing surface (in proper alignment), 
then a copying response can be evaluated on-line against a criterion 
for absolute or average permissible departure from pattern. Such pat- 
terns might include line drawings exemplarizing concepts, letters, or 
words. Given such a terminal, then on-line evaluation of copying 


responses should pose no insurmountable challenge to software designers. 


However, if we allow the response to be free form, then on-line 
evaluation becomes a problem in pattern recognition. On-line system 
evaluation of free form drawing or printing responses should pose a 
considerable challenge for software designers, assuming that hardware 
in present or quickly-attainable form is up to the requirement. To 
the extent that on-line evaluation of free form responses by the 

8 system itself is not presently feasible, we can again fall back on 
the E-augmented system, which will be able to function in the desired 
way if Eis sufficiently quick and celiable. 

Intermediate to copying and free form responses are relational 
responses that reference to a pattern but do not involve copying the 
pattern. An illustration is the diagonal drawing response. A con- 
straining Square or rectangular outline is provided and S is required 
to line a diagonal--with or without a constraint on orientation-- 
through th2 pattern. Similarly, one might provide S with the out- 
line of a human face in front view and require placement of eyes, 
nose, and mouth. In this instance, we could avoid a pattern 
recognition requirement by having responses judged on placement 
rather than shape. Intermediate responses seem as capable of 


@ on-line evaluation as copying responses. 


ss ] 
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Keyboard-referenced drawi response. Imagine a terminal whose 


response area is an x, y matrix of keys elevated above a flat two- 
dimensional surface. Keys should be backlightable by the system or 

by S responding under system-defined conditions and desirably alterna- 
tively backlightable using different colors of light. Such a keyboard 
could be used either to require S to provide matrix-constrained free 
form exemplars of geometric concepts--probably with scale constraint 
imposed--or to complete partially patterned exemplars of such concepts. 
It is apparent that criteria for sequencing peypresses could be applied-- 
Cog,, that the diagonal be constructed deopresetvely left to right. 
Given such a terminal, oneine evaluation of responses appears to 

pose no #nsurmountable hal lenge to software designers. 

Tracking response. | Tracking responses can be referenced to various 
sorts of terminal displays and to various sensorimotor modalities. The 
pursuit rotor tracking reeponse is illustrative. Time on target is one 
measure of pursuit rotor roti clans On-line evaluation of such 


responses appears to pose no insurmountable challenge to software 


designers. 
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Recursive and combinatorial usages of the foregoing categories 
will yield more involved responses. A child asked to order a set of 
Sictures or sentences to tell a story will select an order and perform 
a response sequence that indicates the order selected. “An identifica- 


tion response may take either selected or constructed response form. 


While a time-pressured identification response--e.8-, at tachistoscopic 


levels--may necessitate finer resolution than a CRT provides and perhaps 


shorter interval exposure times than the system finds convenient, the 
required augmenting devices are shelf items and required software efforts 
do not seem undue, Discrimination threshold studies similarly might 
require special peripheral equipment but would turn up no new forms of 
response as responses are defined above--which is in terms of manipu- 
landa, Responses signifying "There it is" or "I see it now" lend 
themselves readily to keypress expression. Where latency of such a 
moteeedepenee adds undesirably to a duration of interest, S's mean 
latency can be evaluated and the system can be required to take this 
out of all such durations. ; 

The test, then, of the taxonomy presented above is not whether one 
can come up with responses whose names or sensorimotor involvements 
are different than the names and molar forms used as exemplars above 
but rather whether one can come up with responses requiring different 
sorts of manipulanda than those cited above. In this connection, 
the tracking response category contemplates various sorts of unnamed 


peripheral equipment--for example, an eye movement camera if visual 


tracking or information searching is to be studied. 
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